Skip to content

Draft: [Coral-Trino] Avoid accidental translation of Trino from_unixtime SQL call#465

Closed
findinpath wants to merge 1 commit intolinkedin:masterfrom
findinpath:findinpath/from_unixtime_function
Closed

Draft: [Coral-Trino] Avoid accidental translation of Trino from_unixtime SQL call#465
findinpath wants to merge 1 commit intolinkedin:masterfrom
findinpath:findinpath/from_unixtime_function

Conversation

@findinpath
Copy link
Copy Markdown
Contributor

@findinpath findinpath commented Oct 17, 2023 •
edited
Loading

What changes are proposed in this pull request, and why are they necessary?

Spin-off from #464 to showcase the solution based on TrinoFromUnixtimeFunction SqlOperator implementation.

Because there is no definition of the function trino_from_unixtime in StaticHiveFunctionRegistry, the test com.linkedin.coral.trino.rel2trino.HiveToTrinoConverterTest#testSubstrWithTimestampOperator fails with the exception:

org.apache.calcite.runtime.CalciteContextException: At line 0, column 0: No match found for function signature trino_from_unixtime(<NUMERIC>)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
	at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:834)
	at org.apache.calcite.sql.SqlUtil.newContextException(SqlUtil.java:819)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError(SqlValidatorImpl.java:4867)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.handleUnresolvedFunction(SqlValidatorImpl.java:1770)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:288)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
	at org.apache.calcite.sql.SqlOperator.constructArgTypeList(SqlOperator.java:593)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:234)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
	at org.apache.calcite.sql.SqlOperator.constructArgTypeList(SqlOperator.java:593)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:234)
	at org.apache.calcite.sql.SqlFunction.deriveType(SqlFunction.java:216)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5626)
	at org.apache.calcite.sql.validate.SqlValidatorImpl$DeriveTypeVisitor.visit(SqlValidatorImpl.java:5613)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveTypeImpl(SqlValidatorImpl.java:1688)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.deriveType(SqlValidatorImpl.java:1673)
	at org.apache.calcite.sql.type.InferTypes.lambda$static$0(InferTypes.java:46)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.inferUnknownTypes(SqlValidatorImpl.java:1868)
	at com.linkedin.coral.hive.hive2rel.HiveSqlValidator.inferUnknownTypes(HiveSqlValidator.java:51)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.expandSelectItem(SqlValidatorImpl.java:474)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelectList(SqlValidatorImpl.java:4104)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3392)
	at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
	at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:1005)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:965)
	at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:216)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:940)
	at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:647)
	at com.linkedin.coral.common.utils.TypeDerivationUtil.getRelDataType(TypeDerivationUtil.java:69)
	at com.linkedin.coral.common.transformers.SqlCallTransformer.deriveRelDatatype(SqlCallTransformer.java:68)
	at com.linkedin.coral.trino.rel2trino.transformers.SubstrOperatorTransformer.transform(SubstrOperatorTransformer.java:65)
	at com.linkedin.coral.common.transformers.SqlCallTransformer.apply(SqlCallTransformer.java:53)
	at com.linkedin.coral.common.transformers.SqlCallTransformers.apply(SqlCallTransformers.java:36)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:52)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:35)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:134)
	at org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:101)
	at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:868)
	at org.apache.calcite.sql.SqlAsOperator.acceptCall(SqlAsOperator.java:123)
	at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:66)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:52)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:35)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:80)
	at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:39)
	at org.apache.calcite.sql.SqlNodeList.accept(SqlNodeList.java:153)
	at org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:134)
	at org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild(SqlShuttle.java:101)
	at org.apache.calcite.sql.SqlOperator.acceptCall(SqlOperator.java:868)
	at org.apache.calcite.sql.SqlSelectOperator.acceptCall(SqlSelectOperator.java:129)
	at org.apache.calcite.sql.util.SqlShuttle.visit(SqlShuttle.java:66)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:52)
	at com.linkedin.coral.trino.rel2trino.DataTypeDerivedSqlCallConverter.visit(DataTypeDerivedSqlCallConverter.java:35)
	at org.apache.calcite.sql.SqlCall.accept(SqlCall.java:139)
	at com.linkedin.coral.trino.rel2trino.RelToTrinoConverter.convert(RelToTrinoConverter.java:91)
	at com.linkedin.coral.trino.rel2trino.HiveToTrinoConverterTest.testSubstrWithTimestampOperator(HiveToTrinoConverterTest.java:573)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
	at org.testng.internal.Invoker.invokeMethod(Invoker.java:701)
	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:893)
	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1218)
	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
	at org.testng.TestRunner.privateRun(TestRunner.java:758)
	at org.testng.TestRunner.run(TestRunner.java:613)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
	at org.testng.SuiteRunner.run(SuiteRunner.java:240)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1170)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1095)
	at org.testng.TestNG.run(TestNG.java:1007)
	at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.runTests(TestNGTestClassProcessor.java:141)
	at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.stop(TestNGTestClassProcessor.java:90)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:133)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: No match found for function signature trino_from_unixtime(<NUMERIC>)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.calcite.runtime.Resources$ExInstWithCause.ex(Resources.java:463)
	at org.apache.calcite.runtime.Resources$ExInst.ex(Resources.java:572)

How was this patch tested?

N/A

@findinpath findinpath mentioned this pull request Oct 17, 2023
Closed
@findinpath
Copy link
Copy Markdown
Contributor Author

findinpath commented Oct 22, 2023

Superseded by #467

@findinpath findinpath closed this Oct 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@findinpath